Intercoder reliability in annotating complex disfluencies

In previous work, we presented an annotation scheme that can describe complex disfluencies. In this paper, we first show the prevalence of complex disfluencies and illustrate the types of distinctions that our scheme allows. Second, we present an annotation tool that allows the scheme to be easily applied. Third, we present the results of a reliability study in annotating complex disfluencies with the annotation tool. We find that subjects, even with a minimal amount of training, achieve high intercoder agreement. This work will help pave the way for speech recognizers to precisely model the structure of disfluencies, both for understanding conversational speech of non-stutterers and for assessing stuttering severity.